Workflow Engine

BREAKING CHANGE for 2020.2 -> 2021.1

The following are breaking changes for the 2021.1 release.

$where in Workflow Engine API Calls

Due to a vulnerability with Mongo, the $where key is no longer supported as part of the query object for several API calls. The $where key is specifically used in Mongo to support a JavaScript expression or function to be used as part of a query. Any APIs that were vulnerable to the usage of $where will now return with an error if $where is included in the query. The following APIs are affected by this, with the relevant parameter noted for each.

Removed APIs

The following APIs have been removed from release version 2021.1 due to instability that could potentially have a negative impact when used improperly. Our usual policy is to provide notice of removal two release cycles in advance; however, in this case, it was necessary that the APIs be removed before the two-release minimum.

Currently getJobsDeep returns a JSON object containing all the task variables (incoming and outgoing) for every single task iteration inside of a job. Ultimately, this can become massive, especially when iterations number into the 1000s or contain significant amounts of data. When this happens, this can lead to a Javascript Heap Out of Memory crash.

What should I do?

Review any custom apps and adapters that might reference any APIs on the list and change them to the replacement call (or remove them if no replacement call is provided).

Since getJobDeep requires a combination of two different calls to retrieve similar data, here is an example of how those APIs can be used together. Previously, getJobDeep would be called via GET /workflow_engine/job/:job_id/deep, and would recursively retrieve all child jobs for the job_id, as well as recursively retrieve all task documents, and then return the job with the metadata combined.

The getJobDetails API only retrieves the requested job, without any task metadata. It can be called using the same parameters as getJobDeep, via GET /workflow_engine/job/:job_id/details. With this API, you can then retrieve the job_id from a child job task and then call getJobDetails again, if the childJob information is needed.

To add task metadata, it is recommended to call searchTasks via POST /workflow_engine/tasks/search with a body similar to the example below, where ":job_id" matches the job_id used in getJobDetails.

  {
    "filter": {
      "job._id": ":job_id"
    },
    "options": {}
  }

This data can then be manually added to the task information from getJobDetails. However, it is not recommended to recursively retrieve all child jobs and child job tasks for a parent job, since this can quickly become a large amount of data.

Workflows using the eventListener task will be migrated to the eventListenerJob task when WorkflowEngine starts. If there is any custom code referencing eventListener it must be updated manually.

Naming Conventions within jobVariables

Naming conventions within jobVariables are now enforced within Workflow Engine. If a workflow with a variable does not adhere to the naming conventions, it will no longer run. This will break workflows when upgrading from the previous version because the name cannot be validated.

To see the rules, review the acceptable Naming Conventions that can be used for values inside of the Itential platform.